Skip to content

Conversation

@damanm24
Copy link
Contributor

Adding more content to our flowey documentation to help ease ramp up for contributors in this area

@damanm24 damanm24 requested a review from a team as a code owner October 27, 2025 21:17
Copilot AI review requested due to automatic review settings October 27, 2025 21:17
@github-actions github-actions bot added the Guide label Oct 27, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a comprehensive developer guide for working with flowey, the OpenVMM project's automation framework. The guide introduces core concepts, architectural patterns, and best practices to help contributors understand and work with flowey effectively.

Key changes:

  • Added detailed documentation explaining flowey's two-phase execution model and backend abstraction
  • Documented core concepts including pipelines, artifacts, nodes, and variables
  • Included practical examples and design patterns for common workflows

@damanm24 damanm24 changed the title Docs: flowey: Add developer/contributor guide for working with flowey WIP: Docs: flowey: Add developer/contributor guide for working with flowey Oct 27, 2025
Copy link
Contributor

@mattkur mattkur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already useful, thanks! I learned a few things reading through this.

Overall, my commentary is primarily stylistic, since the content either already matches what I know, or is telling me things I don't know.

My general style feedback is:

  • Keep the Guide conceptual, and move code and code examples into the rust docs. We publish those as part of the build, so you can even link to them directly from this guide.
  • Use ```admonish rather than the existing Note (for here and other important places)

Besides the semantic questions that I asked, I also wonder:

  • Do we have code layout or naming conventions? (e.g. _jobs folder, artifact_*.rs for artifact definitions)
  • How much processing should I do in an artifact? Is it just a record of files? Or should I ever do a transformation in one?

Copy link
Contributor

@daniel5151 daniel5151 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smalis-msft sent this my way, and given that I didn't end up sending in flowey docs before I left msft (tbf, I had a very short runway prior to leaving!), I figured the least I could do is chime in with some comments here.

maybe the most important feedback is that you def want to invert the presentation of a lot of the concepts in the guide. start from the base building blocks (steps), go into how those compose into nodes (with requests), how those get coalesced into jobs, and how those become pipelines (with a discussion on artifacts). As written today - this guide is really hard to follow, unless you are willing to read this whole thing multiple times.

also, having a high-level overview discussing the "why" behind flowey seems important to set the stage for the rest of the discussion. i.e: talk about how traditional back + YAML based pipelines are impossible to maintain, rely heavily on non-local reasoning and global state, can't be run locally / ported between platforms, etc...

@github-actions
Copy link

@damanm24 damanm24 changed the title WIP: Docs: flowey: Add developer/contributor guide for working with flowey Docs: flowey: Add developer/contributor guide for working with flowey Nov 3, 2025
@benhillis benhillis requested a review from Copilot November 5, 2025 19:09
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 11 out of 12 changed files in this pull request and generated 1 comment.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

**[Pipelines](https://openvmm.dev/rustdoc/linux/flowey_core/pipeline/trait.IntoPipeline.html)** are the top-level construct that defines a complete automation workflow. A pipeline specifies what work needs to be done and how it should be organized. Pipelines can target different execution backends (local machine, Azure DevOps, GitHub Actions) and generate appropriate configuration for each.

**[Jobs](https://openvmm.dev/rustdoc/linux/flowey_core/pipeline/struct.PipelineJob.html)** represent units of work that run on a specific platform (Windows, Linux, macOS) and architecture (x86_64, Aarch64). Jobs can run in parallel when they don't depend on each other, or sequentially when one job's output is needed by another. Each job is isolated and runs in its own environment.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each job is isolated and runs in its own environment.

Not really true, yes they have some rust-level isolation, but they're still running on the same system and could rely on implicit global state that you forget to track. It's not like we're doing any really heavy sandboxing.

@smalis-msft
Copy link
Contributor

This looks fantastic, thanks for taking this on! Just a few little cleanups and I think it's good.

@github-actions
Copy link

@damanm24 damanm24 merged commit 1b651d5 into microsoft:main Nov 11, 2025
80 of 82 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants